Data Sets and Stimulus Sets
Mixed Hybrid Search dataset
Incidental findings are a significant issue in clinical radiology. They are difficult to study in a systematic fashion because both the stimulus material and the observer population are hard to assemble. Here we propose a “model system”, in the form of a ‘mixed hybrid search’ task. It can be used to investigate the fundamental cognitive processes that lie behind incidental finding errors. Once those principles are identified, tractable experiments can be designed for clinical settings with expert observers.
In the ‘mixed hybrid search task,’ observers search for any of three specific targets (e.g. this rabbit, this truck, and this spoon) and three categorical targets (e.g. masks, furniture, and plants). The hypothesis is that the specific items are like the specific goals of a real world search and the categorical targets are like the less well-defined incidental findings that might be present and that should be reported. In all of these experiments, varying target prevalence, number of targets, etc., the categorical targets are missed at a much higher rate than the specific targets. This paradigm shows promise as a model of the incidental finding problem.
More details of the experiments stimuli and methodology are presented below. The behavioral results of the experiments are recorded in excel spreadsheet and are available for downloading by clicking on the appropriate hyperlink at the bottom of the page. If you use the data, please tell your readers where you got it. We would be grateful to know about any interesting facts that you might learn from the dataset. Please send us a note (jwolfe@bwh.harvard.edu).
Stimuli detail and methodology
All stimuli were photographs of isolated objects (provided by Talia Konkle). Available categories were: Animals, Cars, Hats, Masks, Shoes, Fruit, Furniture, Kitchenware, Musical Instruments, Plans, Rocks and Minerals, Signs, Sweets, Time Pieces, and Weapons. For each of the fifteen categories, we had 50 images.
At the start of each block, observers learned the memory set by viewing each item and/or the name of the category for 3 seconds. After passing a memory test, observers performed 30 practice and 300 experimental visual search trials. In Experiment 1, observers searched for specific items, categorical items, or both, in the critical, mixed condition. There were six blocks of trials, differentiated by the memory set of possible target items. Observers could be asked to look for any of three specific target types, three categorical target types, six specific, six categorical, or a mixture of 3 specific and 3 categorical target types. Experiment 2 repeats the mixed hybrid search experiment with a systematic variation in the prevalence of the specific and categorical targets. The relative proportion (categorical/specific) of each type of target varied across three conditions: 20/80, 50/50 & 80/20. In Experiment 3, the presences of specific and categorical targets on a trial were independent of each other. This raises the possibility of multiple targets on each trial.
Datasets
One dataset for each experiment is available for downloading. Experiment 1 & 2 have 20 labeled fields each. Experiment 3 has 31 labeled fields.
- Memory Set Size refers to the number of items that was shown to the observers at the start of each block (3 or 6). The observers were instructed to learn the memory set.
- Visual Set Size is the number of items on the screen during the visual search task (4, 8 or 12). The observers were asked to search for the target within the visual set.
- Target Name is the name of the target image file. ‘Absent’ is recorded if no target was presented in a trial.
- RT Filter indicates the RTs that were removed from analysis. RTs were considered as ‘TOO SHORT’ if they were less than 200 msec, ‘TOO LONG’ if they were greater than 10,000 msec.
- Message explains the outcome of the trials (and reproduces the information given to the subject). Specifically, trials are labeled as HIT (target present correct trials), TNEG (True negative -trial absent correct trials), MISS (target present incorrect trials), FA (false alarms).
Download
Experiment 1: Mixed specific and categorical targets [.excel file]
Experiment 2: Varying the relative prevalence of specific and categorical targets [.excel file]
Experiment 3: Independent specific and categorical targets [.excel file]